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ABSTRACT 

This paper documents the general findings and recommendations of the Design for Safety Program s Study of the Space 
Shuttle Program s (SSP) Problem Reporting and Corrective Action (PRACA) System. The goals of this Study were; to 
evaluate and quantify the technical aspects of the SSPs PRACA systems, and to recommend enhancements addressing 
specific deficiencies in preparation for future system upgrades. The Study determined that the extant SSP PRACA systems 
accomplished a project level support capability through the use of a large pool of domain experts and a variety of distributed 
formal and informal database systems. This operational model is vulnerable to staff turnover and loss of the vast corporate 
knowledge that is not currently being captured by the PRACA system. A need for a Program-level PRACA system providing 
improved insight, unification, knowledge capture, and collaborative tools was defined is this study. 
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1. INTRODUCTION 

The Space Shuttle Program s (SSP s) Problem Reporting and Corrective Action (PRACA) systems and their supporting 
infrastructure (used to report discrepancies, non-conformances, problems, track engineering dispositions, corrective actions 
and provide data for trend analysis and reporting) are an essential tool for managing Shuttle safety and readiness for flight. . 
The charter document that describes the requirements for the SSP PRACA system is NSTS 08126, Revision G [ref. 2], 
Because of their importance, the PRACA systems have been the subject of several recent reviews aimed at improving the 
systems utility and improving the motivating requirements. In September 1999, NASA chartered the SIAT to provide an 
independent review of the Space Shuttle sub-systems and maintenance practices. The SIAT published its report in March 
2000 [ref. 3]. In the SIAT report, several findings and recommendations were raised specifically regarding the SSP problem 
reporting practices and systems that may adversely affect Shuttle safety. 

The NASA Jfcbnson Spec* Center {JSQPRACA Evaluation Teanr^PET) ww c reated to adtfaw ; 

recommendations from the Shuttle Independent Assessment Team (SIAT), the initial NASA Ames Research Center (ARC) 
assessment comments, and other SSP sponsored PRACA audits and reviews. The PET was established by SSP Review 
Control Board Action S060341R5(3-l) [ref. 1]. The PRACA Enhancement Pilot Study (the subject of this paper) was 
coordinated with the JSC PET, and as part of a new NASA Initiative — the Design for SafetyProgram (DfS). A team at 
NASA Ames Research Center performed the PRACA Enhancement Pilot Study. 

This paper documents and provides a general overview of the technical evaluation of the existing operational SSP PRACA 
systems. The evaluation then generalizes the technical findings and recommends enhancements to improve this critical 
NASA distributed information system. The full results of the Study and the detail assessments of the above four areas are 
documented in the NASA Ames Research Center Technical Memorandum published on the Study [ref. 4]: 

• User Interface 

• Database and Data Management 

• Network and System Architecture 

• Problem Reporting Work Processes 

A key element of the continuing success of the Space Shuttle Program and the operation of the multiple PRACA systems has 
been the dedicated and enthusiastic staff of NASA and its contractor team. The progress of this Study was greatly aided by 
the tremendously dedicated and hard working individuals supporting the Space Shuttle Program. Everyone we spoke to 





through the course of this Study was highly cooperative and willing to assist us in completing this assessment and to ensure a 
continued safe Space Shuttle System. 

The broad general assessment of the current PRACA systems is that they achieve sufficient project support capability 
through engagement of a large number of highly dedicated and competent staff The systems are frequently redundant and 
not interconnected which produces inefficiencies and the potential for data loss and input error. The current approach 
employed in the PRACA deployment does not scale or adapt easily to changes in workforce or technology. Further, the 
expert knowledge that is required to utilize the PRACA systems is not captured or documented. As a result, the current 
PRACA system is not capable of supporting risk-assessment functions performed at the Program- level. 

We believe that an Agency-wide NASA/Industry team in conjunction with the SSP PRACA workforce can bring together the 
required expertise, knowledge, and advanced IT capabilities necessary to achieve NASA s Information Management vision 
for PRACA. In so doing, PRACA will remain a critical and vital system, enabling a reduction in the nsk and improvements 
in safety while supporting the Space Shuttle Program into the next decades. 

2. MOTIVATION AND APPROACH 

The Creation of the SSP PRACA System 

In 1987, after the Challenger accident and in response to the Rogers Commission recommendation to provide NASA Space 
Shuttle management and decision makers with readily available, timely, and accurate data, the Program Compliance 
Assurance and Status system (PCASS) was formed. The PCASS is defined in document System Integrity Assurance 
Program, NSTS 07700 Volume XI, [ref. 6] 

The NSTS 07700 goals were to impact Shuttle processing, safety, and readiness for flight by enabling continuous process 
improvements. The premise was that an overall PRACA System would allow users access to the current and historical data 
necessary to perform trend analysis and reporting to aid in the process planning and improvement. Currently, to provide the 
data necessary for this Program-wide view into the PRACA data, a combination of paper records, on-line databases from 
separate PRACA systems, and corporate/expert knowledge and skilled personnel are required. 

PCASS is being re-hosted from the main frame -based PCASS onto a Unix server with a browser interface. However, the 
NSTS 07700 goals of an interactive Shuttle data store for use in trending, safety and reliability analyses are not yet being 
realized 

Assessment Approach 

The objective of the Study was to document a quantitative assessment of the technical and operational status of the SSP 
PRACA systems and elements. In addition to the as-implemented aspects, the team desired to understand the as used 
issues and challenges with the PRACA systems so that any recommendations, while technically feasible, can also be 
evaluated for their practicality and work environment utility. 

The approach taken by the Study team was to interview, understand, and assess. This approach required multiple site visits, 
telecons, and interviews with as many of the people involved in the PRACA system as possible (managers, users, customers, 
etc). The team consistently noted the support and cooperation by the NASA and contractor staff throughout the SSP PRACA 
system. This was fairly unique in the team s experience to see such cross-center cooperation and enthusiasm for progress 
towards a common goal. All of the team s requests for information, documentation and time were professionally addressed 
and met the team s needs. 

The quantitative technical assessment began in March 2000. The Team visited and interviewed PRACA systems owners and 
users at JSC, KSC, and MSFC. The purpose of these meetings was to understand how the SSP elements collect, manage, and 
use the problem reporting data. The team also interviewed multiple safety, reliability, quality and mission assurance users of 
the PRACA data to determine the desires and implicit requirements for the PRACA systems. As part of the interview 
process, the team collected available system documentation recommended by the contacts. 


3. STATE OF THE SSP PRACA SYSTEM 


This section details the state of the SSP PRACA systems. This information was collected from the interviews with various 
PRACA system owners and the Project/Element level and analysis of available PRACA documentation. 

PRACA System Overview 

The NSTS 08126 Revision G document was signed on February 2, 1996. That document provides the minimum 
requirements and responsibilities applicable to all SSP PRACA systems, as required by NHB 5300. 4( ID 2), Safety, 
Reliability, Maintainability and Quality Provisions for the SSP [ref. 7]. The objectives of NSTS 08126 Revision G are to: 

a. Establish uniform standards to ensure safety, reliability, and quality of SSP. 

b. Establish the requirements/procedures to assure problems are dispositioned prior to flight. 

c. Ensure appropriate corrective action is taken in a timely and cost effective system. 

d. Provide the problem data necessary to support engineering analyses and logistics management. 

The NSTS 08126 Revision G does not address the issues or requirements for extracting the data necessary for trend analysis 
and reporting. While it does provide the data for engineering analyses and logistics, this has not proved sufficient to the 
safety analysts and has necessitated having adjunct data sources and databases to perform risk and reliability trend analyses. 

NSTS 08126 Revision G has established some uniform standards for ensuring safety, reliability, and quality of SSP, but has 
not required or documented the critical information and data standards. These include data management standards, such as 
field naming and database schema, that would enable the development of common SSP-wide PRACA data systems. Such a 
system is discussed in the recommendations, and is what the current implementation of the SSP PRACA lacks. The use and 
design of the existing PRACA systems, described in the following sections, points to the innate desire of the SSP for a 
unified access to PRACA data. 

Implementation of SSP PRA CA 

The current SSP PRACA System is a collection of computer hardware, software, networks, and databases, as well as 
extensive paper files distributed across several NASA, USA and other contract support sites. These separate PRACA systems 
are managed by various teams of individuals whose job is to maintain the systems, keep the databases current, and assist in 
the extraction of data for trending and reporting. One of the key components contributing to the success of the current 
PRACA systems is this support staff that forms the extensive corporate Mid institutional knowledge necessary to analyze, 
reduce, produce conclusions, and report results from the PRACA databases. Without these highly trained and expert stiff, the 
utility of the PRACA data is reduced significantly. 

The data management capability behind the SSP PRACA System is a conglomeration of Project-level database systems. 
Many of the component systems are designed as workflow management tools with strict requirements for streamlining 
accurate and timely problem resolution. Other systems are focused upon capturing the problem report data at a high fidelity 
useful in safety analysis and data mining applications. It is these two dissimilar requirements (workflow efficiency vs. 
extensive problem documentation) that are at odds in the current SSP PRACA systems. 

For the purposes of SSP use, the PRACA system is viewed as a collection of domain-expert managed systems. At the time of 
this Study most of the data from all but one of the Project PRACA systems could be queried through a centralized data 
warehouse for summary and condensed report viewing. This warehouse is called the Advanced Data Acquisition and 
Management (ADAM) [ref. 7J. However, as noted by the SIAT, ADAM cannot be data mined or navigated by the non-expert 
user. As a result, the SSP uses the PRACA systems principally by contacting the appropriate responsible domain experts and 
requesting that a report or data trending exercise be produced for the Program office. Then, a team of domain experts at JSC, 
KSC, MSFC and other support sites accesses the PRACA component databases, other experts, additional off-line databases, 
and paper records to produce a report for SSP. These reports are usually placed in the PRACA systems and become part of 
the on-line record. It is important to note that reports of any sophistication (i.e., data mining, detailed cross PRACA 
correlations etc.) cannot be generated in real time using the on-line PRACA systems alone. 


4. PRACA SYSTEM GENERAL FINDINGS 


Upon completion of the on-site interviews, reviews and technical assessments of the multiple PRACA systems, the team 
identified a set of general findings, and a set of findings specific to the four technology areas identified in our Approach. The 
overall assessment of the existing PRACA systems is that they are inefficient and potentially vulnerable to data loss and input 
error. The current approaches do not scale or adapt easily to changes. The expert knowledge that is required to utilize the 
PRACA systems is not captured or documented. Overall, the existing PRACA systems are incapable of supporting Program- 
level risk assessment. These general and specific technical findings are discussed in the following sub-sections. 

Current PRACA Does Not Meet SSP Needs 

The PRACA systems in use by the SSP are not sufficient to meet the SSP current and future needs (as expressed by NASA 
SSP management, the SSP requirements documentation (i.e., NSTS 07700 Volume XI), and from a sustainable workforce 
perspective). This general finding is primarily due to lack of clarity in expression of a vision for PRACA, which is the result 
of three primary causes. 

1 . The SSP is currently able to accomplish its PRACA Shuttle management tasks, yielding the perception that 
PRACA works. Because of this perception, questions addressing the essential functional, systematic and 
architectural aspects of PRACA are not being asked. 

2. The SSP requirements documentation does not preclude the use of data outside the formal PRACA data systems. 
This enables a capability much greater than would be possible if the users were restricted to PRACA system data 
only. As a result, the SSP has not been confronted with the data limitations of the current PRACA systems. 

3. The current SSP PRACA system is perceived as having a system-wide, user-navigable data warehouse capability. 
This perception is further reinforced by the use of domain experts who extract data and reports from the various 
Project/Element-designed and supported systems. This gives the impression of a system-wide data mining and 
navigation capability. Because of this perception, the SSP is not articulating and advocating a data warehouse re- 
architecture to PRACA (believing it already has one). 

These three reasons are described further in the following sub-sections: 

The Perception That PRACA Works. 

The premise of the PRACA capabilities described in NSTS 07700 Volume XI sections 3.4 and 4.1.5.x is to integrate 
program element trend systems, perform analysis, and provide data formatted for management visibility to support Shuttle 
Safety Assessments. The SSP does this by reliance upon subsystem domain experts, and the performing safety organizations 
skilled personnel, to report status and trends to the Program office. The domain experts possess substantial tetitutto&al 
knowledge and are able to draw from information outside the PRACA data systems. The reports provided to the SSP 
therefore are based on large amounts of data outside of the PRACA systems. The loss of the domain experts and the external 
data sources would significantly degrade the quality of the PRACA reporting and trending. The use of domain experts 
enhances the quality of the knowledge extracted from the PRACA systems, giving the Program Office the mistaken 
impression that the PRACA systems alone possess equivalent data and knowledge-generation capabilities as those presented 
in the reports. 

For the trending and analysis groups to perform their tasks using current PRACA, it is sometimes necessary to filter or 
correct the data and generally augment PRACA data to produce meaningful results. Several NASA safety, reliability and 
quality assurance (SR&QA) groups identified this practice as necessary to perform meaningful trending and analysis for the 
Program office. The JSC Shuttle Orbiter SR&QA group, for example, created and manages the Shuttle Risk and Reliability 
Analysis Database (SRRAD) and created manual and automated filtering and data processing routines for this purpose. The 
SRRAD database and data filtering and correction process are examples of data and expertise upon which the SSP reports are 
dependent but are not part of any of the formal PRACA systems. 

The Use of Non- PRACA Data in Reports 

The use of data outside the formal PRACA system has allowed the evolution of informal data systems upon which the SSP 
depends. These informal databases exist, in many cases, at the expense of a proper upgrade of the formal PRACA systems. 


The base SSP PR AC A requirements documentation {NSTS 07700) imply that a PR AC A System will be the official 
repository for all data necessary to manage Shuttle problem reporting and corrective action assessment, diagnosis, and 
trending. However, the NSTS 08126 Revision G requirements also turn responsibility for generating the status, assessment 
and diagnosis over to Project; Element domain experts with no constraints on the data they are permitted to use to generate the 
reports. In order to produce the best reports possible, the experts naturally draw from all the information they can access. 

The dependence of the SSP on domain experts to produce reports is codified in the requirements document (i.e., it is the 
responsibility of the domain expert centers to manage their subsystems and report their data to the program office). This has 
effectively hidden the lack of stand-alone capability the SSP desires for PRACA. It has also hidden the fact that the PRACA 
system cannot be data mined by the non-expert users. The expert users can manually navigate the various information 
systems using implicit and innate familiarity with the data, thus giving the appearance of a data mining capability. A new or 
inexperienced user would not have the knowledge or context to understand the syntax, or codes used for each of the PRACA 
systems. Indeed, an expen on one SSP PRACA system is not an expert on all PRACA systems. 

As with any large requirements-based information system, the domain experts have found that it is easier to build and 
manage an informal database to augment PRACA data outside of the SSP PRACA purview than it is to upgrade the SSP 
PRACA databases. For example, as mentioned in the section above, the JSC SR&QA group performs trending and analysis 
for the Program office by creating and managing the SRRAD database. The SSRAD database is used in conjunction with the 
PRACA data but is not part of any formally recognized PRACA system. These adjunct PRACA data sources and their 
requirements need to be addressed in the overall SSP PRACA picture. 

The Perception that PRACA is a Navigable Warehouse of Data 

The SSP has expressed wishes to incorporate the individual PRACA systems into a unified program data warehouse but has 
not initiated a major re -architecture or training program to elevate the focus from the SSP Element/Project level to the 
Program level. The Project-level or component PRACA systems resident at JSC, KSC, and MSFC have been designed and 
implemented by the PRACA domain experts at each center. These systems were developed to meet their individual task and 
project support needs as stand-alone systems. The inevitable local prioritization of tasks results in no two systems having the 
same global motivation for existence or the same technical implementation basis. The Project-level systems are not innately 
amenable to hybrid data warehouse architecture. 

USA s ADAM is a good first attempt to provide unified PRACA data and some associated information. The ADAM effort 
sidesteps the individual Project PRACA system differences by duplicating the data onto one site, where an interface and 
mapping can insulate the user from system-to- system navigation difficulties. ADAM cannot, however, resolve the issues of 
data quality, consistency, integrity, and breadth, which arc limited by thc^source PRACA systems. ADAM is mertising data 
visibility and bringing attention to data errors and data field mapping inconsistencies across the PRACA systems. ADAM is 
not addressing the system-wide architectural changes required of the Project systems. 

Much of the data the Program requires and desires from the PRACA systems to make them more useful for Safety and Risk 
analysis are not consistent with the current Project/Element focus. Data currently gathered for the Program that is not 
necessary for local domain needs is generally handled with less care because its value and use are locally subjective. This is 
important to note for a number of reasons. It can lead to incorrect entries because the person making the entry does not 
understand the purpose of that data field in the PRACA form. Additionally, end users such as people doing trending and 
analysis may not understand the context in which the data were acquired and therefore why fields are being filled out in 
particular ways at operational sites. These mutual misunderstandings can produce incorrect data, which degrades the validity 
of any trending analyses using these data fields. The main commonality between the PRACA systems comes from the NSTS 
08126 document. While each of the various PRACA systems comply with the current 08126 revisions, the Project-level 
PRACA systems have there own guiding requirements and documentation, of which the 08126 requirements play only a 
partial role. Data field naming conventions are solely left to the Project to implement in their local databases. 

Streamlining efforts (efficiency, turn-around time, work-flow) at the Project and domain level tend to resist addition of fields 
that impede the streamlining efforts. This is due in part because the local PRACA system is often used for a number of 
functions besides problem reporting and corrective action. Thus many individuals collecting the data may not see their job as 
connected to the SSP system-wide PRACA effort. This also contributes to opacity and misunderstandings of the roles and 
purpose of the SSP PRACA systems. 


No PRACA System-wide Philosophy 


There is a lack of a clearly expressed vision for SSP PRACA and subsequent lack of a system-wide buy-in to the PRACA 
System philosophy. While the Space Shuttle Program expectation for PRACA is as a part of the PCASS. many view the 
PC ASS as one of the several systems comprising PRACA data sources rather than as a PRACA end design goal. The 
problem was further complicated by a lack of clearly acknowledged PRACA owner For example, none could be identified 
or established during a visit to the SSP office in January 2000. Since January a PRACA owner has been identified, within the 
SSP as well as for the USA contract, by the JSC PET activity. 

The creators or Project-level managers of the PRACA systems accept the SSP system-wide data resource goal but do not yet 
view PRACA data as a Program-wide resource that can be controlled and managed by the SSP office. An overall PRACA 
System as an organizational or technological entity does not exist, and was not required by the NSTS 08126 Revision G 
document. The JSC PET has rewritten and enhanced the NSTS 08126 document to a Revision H and this is discussed later. 

Effect on PR ACA Oata 

The data collection workforce is not currently trained in PRACA system Program data needs and usage. The data collection 
complexity imposed on the local (domain) PRACA teams is not a primary consideration addressed in the PRACA 
requirements documentation. Because of the Project-focused development of the individual PRACA systems and their work 
practices, the resulting multiple definitions, levels and functions of PRACA lead to opacity between parts. That is, each user 
of a PRACA system understands it from the viewpoint of their Project/Element and their task. There is not a training process 
enabling a universally shared understanding about what PRACA data is, what its levels are, who has responsibility for which 
parts of PRACA data, and how to weigh the priorities of Program data collection against local task schedules and deadlines. 

We found that Project users, the engineers and technicians who originate repons, were more likely to understand PRACA 
data from the perspective of their center, or even of their specific job and its procedures, without understanding the larger 
implications for how the system(s) works and the SSP-wide service that it provides. This can lead to tensions between the 
functions, and to the possibility of bad data being entered into the system. 

For example, technicians at KSC who have found a nonconformance are required to fill out a PRACA Problem Report. 
Interim Problem Report or Discrepancy Report as a pan of the process for dealing with nonconformances. However, the 
report s central use for them is to schedule the work of repair and assignment of safety constraints on other work, which may 
not be performed until the nonconformance is dispositioned. Many of the required data fields in the problem report form that 
are relevant for problem reporting are irrelevant to the scheduling process, and may be seen by users as demanding 
information to which they do not have ready access or which requires too much valuable time to answer. 

One example of such a field is Vendor . It may be useful for developing trending data at the Program level, but this use is 
not clear to technicians. The field is not necessary for any of the work that the technicians do, nor is it particularly valuable 
to the engineers and schedulers who direct the technicians work. This confusion can lead to technicians filling in any value 
that they know will pass the inspection process, rather than attempting accuracy. 

Effect on Future Potential for PRACA 

The lack of a clearly articulated and adopted vision across the PRACA systems has an effect on the future potential for safety 
and risk analyses. Many of the visions expressed by NASA senior management for the ideal PRACA System include a 
prognostic capability with data search, navigation, and mining that extends across the Shuttle Program. Without some effort 
to promote unity amongst the systems or their data, the potential technical leverage from similar systems (e.g., the Aviation 
Safety Program) and NASA research Programs such as Design for Safety will be reduced. There are many opportunities for 
interaction and data sharing with the digitized Shuttle components databases, commercial aviation maintenance planning and 
scheduling systems, and model-based reasoning systems that could significantly enhance the utility of the PRACA data for 
safety and risk analyses within the SSP. This is discussed further in the recommendation section of this report. 


NSTS OS 1 26 Revision H Updates 


The JSC PET has completed much of a rewrite and update to the NSTS 08126 document. This revision is appended as 
Revision H to identify it as the successor to Revision G. Revision H better reflects the desired scope and global functions 
expected of the SSP s PRACA system. The Space Shuttle Program Review Control Board is expected to approve 08126 
Revision H in the summer of 2000. 

Revision H now states that the goal of the PRACA system is to establish a process to continuously improve the safety and 
reliability of Space Shuttle hardware, software, and critical ground systems. The PRACA system will provide the SSP and 
all SSP Elements/Projects: 

1 ) Accurate and immediate visibility into problems; and 

2) An accurate historical database to support problem trend analysis, provide failure history, support anomaly investigation, 
and to document corrective actions. 

Revision H also recognizes that PRACA is only useful if the reported information is accurate and correct It emphasizes that 
sufficient attention must be paid to insuring accuracy of the data comprising the problem report, failure summary, root cause 
analysis, and in/out-of-family screening. 

NSTS 08126 Revision H defines and enhances the SSP requirements for problem reporting, analysis, disposition, resolution, 
and trending. Problems that are documented in PRACA include: Space Shuttle hardware, Orbiter software discrepancies, 
main engine software discrepancies, Launch Processing System, Ground Support Equipment and Launch & Landing facilities 
that support mission to mission processing of flight hardware. The Revision H document establishes: 

1 ) Uniform criteria for reporting problems; 

2) Requirements for problem disposition and closure; 

3) Requirements for documentation of corrective action; 

4) Requirements for problem documentation to support engineering and trend analysis; 

5) Requirements to support logistics management; and 

6) Definition of problem report data elements and terminology. 

The NSTS 08126 Revision H now addresses the requirements for extracting the data necessary for trend analysis and 
reporting. In addition, this latest version of the PRACA system requirements greatly improves and clarifies the requirements 
for the PRACA systems. Work is continuing on the PRACA data element definitions and establishing the database code 
translation tables to enable some mapping between the various PRACA systems data. These definitions and tables are to be 
included in the final version of NSTS 08126 Revision H. 

5, RECOMMENDATIONS 

In order to recommend modifications, upgrades, and enhancements to the SSP PRACA systems, we must establish two 
things: first, what PRACA currently is; and second, what PRACA should be. This study has endeavored to identify the 
current state of PRACA (i.e., what PRACA currently is). As for what PRACA should be, there are three fairly distinct mental 
pictures emerging from the team s interviews with the PRACA workforce, SSP management, and NASA senior management. 
These are: 

1. The Proicct/EIement domain expert view : 

PRACA should remain a collection of relatively simple databases that support the work process and record-keeping 
functions. These databases are designed primarily to support the domain experts who are responsible for reporting 
Project/Element status and trending to the SSP. The domain experts would prefer that the SSP continue to rely upon the 
domain experts for data extraction, filtering, analysis, interpretation and reporting from the PRACA databases and other 
sources. 

2. The Fund Source (SSP/NASA s Human Exp loration Enterprise) view: 

PRACA is a multi-center data system that is vital to the SSP mission. The domain experts role in the PRACA system is 
consistent with the team problem resolution approach and is not seen as a potential problem. Ongoing reviews and 
relatively stable workforce will sustain the system s viability into the future. Additional work on PRACA should be 
justified based upon new capability. 



3. The NASA Information Management view; 

PRACA should be a state-of-the-art data warehouse capable of data mining and advanced data analysis and trending 
using a simple and uniform point and click interface. The system should preclude data errors, incomplete problem 
tracking, and catch potential problems that might otherwise go undetected (e.g., escapes and diving catches ). The 
system should reduce the sole dependence on domain experts and corporate knowledge, placing the power of top-level 
knowledge and information in the hands of anyone with access to the system via a simple user interface. Additionally, 
advanced data mining capabilities would support the SR&QA analysts to improve the speed and accuracy of their 
assessments. With the Shuttle expected to fly another 25 years, the system architecture must be dynamic and capable of 
overcoming changes in workforce, technology, and flight rate. The system should be enhanced to provide a foundation 
enabling the future implementation of a safety and nsk prognostics capability. The system should serve as a model and 
pathfinder for the Agency. 

As we have noted previously, there was no overall SSP PRACA owner until recently and there still is no clear vision 
declaration for scope and functionality. The general vision of capability proposed in the NSTS 07700 volume XI is not being 
fulfilled with the present PRACA systems. 

The Study team has chosen to present its recommendations with the assumption that the NASA Information Management 
view of the PRACA system should serve as a goal for the final system state and our recommendations. Sensitivity to the 
Project and Fund Source views was maintained but as secondary considerations. Given this attention and assumption 
guideline, this Study has identified several recommendations for modification, upgrade, and enhancement of the SSP PRACA 
System. The general recommendations are discussed as follows: 

Global Perspective 

The SSP-identified PRACA owner needs to make a global assessment of PRACA with both a short-term and long-term view. 
It is important to answer vision-defining questions such as: 

• Is PRACA sufficient for the SSP needs? If so, for how long? 

• Is PRACA to be a cutting edge information management system? Is it to serve as an example for Agency emulation? 

• Is PRACA to look beyond SSP focus to leverage other safety and reporting systems? (Aviation Safety Program, 
Commercial aviation scheduling and planning systems, model-based reasoning systems, digitized Shuttle systems, 
other NASA PRACA systems, etc.) 

• What is the evolving role for PRACA looking into the next 25 years? 

• W'hat is the relationship of PRACA to the changing NASA workforce? And how does that impact PRACA 
functionality over time? 

• Is PRACA to be the foundation of a Safety and Risk Prognostics System for the SSP? 

It is equally (if not more) important to answer design questions driving the requirements such as: 

• Who are the customers for the system s data? 

• Who are the users of the system? 

• Who are the managers of the system? 

• What skill level(s) is expected of the owner, customers, users, and managers of the system? 

• What is the security level of the data in the system, and what is the desired visibility in the community? 

• How large a dependence on expert knowledge and human interpretation is acceptable? 

• Is it permissible/desirable to use data outside of PRACA (and PCASS) or should PRACA be the sole source of data 
access? 

• What are the roles of the Program office as an owner, user, and a customer of PRACA information? 

• What is the role of PRACA at the data collection level? At the Program/Element level? 

Once these and other similar questions are answered, the SSP should clearly articulate its vision and train and/or inform 
personnel in all of the levels of the PRACA system. 


PRuACA as an Element of Safety and Risk Prognostics 


One of the unrealized possibilities tor the PRACA database systems is as a foundation for a Safety and Risk Prognostics 
capability. Prognostics in this sense go beyond simple data trending, to provide a true predictive capability that could greatly 
enhance the decision-making capabilities of the SSP and the safety of the Shuttle. 


• Establish a plan for PRACA system evolution that will enable the development of a future Safety and Risk 
Prognostics capability. 

Impact: 

• Improve the breadth and depth of the SR&QA analyses performed by the expens in a given time frame, as well as 
ensure the high quality of the PRACA data for such analyses to be made. 


• Provide a manager-level overview and quick look assessments of Shuttle safety and risk data, 

- Enable SSP management to be more proactively involved and up-to-date on the performance and safety 
trade-offs for the Shuttle fleet. 
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Figure 1 - PRACA Data Use Concept 


As noted in the figure above, a technology gap exists in the current PRACA technology pyramid. This technology gap is 
currently compensated for by the use of domain experts to manually search, interpret, filter, and process the data into 
knowledge for the SSP. This technology gap should be eliminated by: 

1. Implementing improvements to the PRACA systems in the fundamental enhancements areas as shown in the figure 

2. Implementing advanced data access, data mining, and unified user interfaces. 

We believe that an improved and enhanced PRACA System could radically improve the breadth and depth of the SRQ&MA 
analyses performed by the experts in a given time frame, as well as ensure the high quality of the PRACA data for such 
analyses to be made. Additionally, the future PRACA System would provide a manager-level overview and quick look 


assessments of Shuttle safety and nsk data. This will enable SSP management to be more proactively involved and up-to- 
date on the performance and safety trade-offs for the Shuttle fleet. Specific recommendations to do this are addressed in the 
Study s NASA Technical Memorandum. 

Update of PH4CA Requirements 

Since the delivery of the SI AT report in December 1999 and the initial ARC PRACA review in January 2000, the SSP 
created the PET to reassess and revise the NSTS 08126 requirements for PRACA. In conjunction with this activity, the USA 
Integrated PRACA Team has addressed many of the underlying processes and motivations for the various PRACA systems 
under its control. These activities do several important things: 

• They unify some of the reporting requirements. 

• They enhance some of the access requirements. 

• They respond to multiple SSP and USA audits and reviews and address the SIAT report concerns and the informal 
ARC Study comments. 

It is also critically important to answer the questions driving the requirements that have not been completely addressed by the 
aforementioned SSP and USA activities. 

PRACA Owners 

The creators or Project-level managers of the PRACA systems do not yet view PRACA data as a program-wide resource. An 
overall PRACA System as an organizational or technological entity does not exist, and was not required by the NSTS 
08126 Revision G document. The JSC PET has rewritten and enhanced the NSTS 08126 document to a Revision H. 
Revision H better reflects desired scope and global functions required of the SSP s PRACA system. The Space Shuttle 
Program Review Control Board is expected to approve Revision H in the summer of 2000. 

The SSP has identified a Shuttle Program PRACA Owner and USA has identified an internal PRACA owner. The team 
recommends that these owners take the action to declare the vision for the PRACA System and its evolution over the next 
25 years. The vision declaration should create a concrete image in the minds of the PRACA workforce, from the data 
collectors through the SR&QA analysts to the SSP management office. 

Program- Level Access 

As we discussed in a previous section of the study, the existing PRACA System is not sufficient for Program-level data 
mining and SQ&MA assessment. The SSP currently meets the necessary constraint of having enough problem reporting data 
and insight by relying on a set of domain experts possessing extensive knowledge of the Shuttle subsystems and the PRACA 
data, and who have access to additional non-PRACA (formal) data. These experts produce consolidated repons and 
summaries for the SSP office from which the SSP performs its tasks and formulates decisions. This sole dependence on 
expert knowledge and domain experts has shielded the SSP office from several PRACA deficiencies, including: 

• PRACA data alone does not provide enough information for Program-level trending and data mining applications. 

- There are multiple data sources on maintenance, repair, corrective actions and engineering dispositions (CARs, 
hazard reports, engineering databases, expert knowledge, etc.) not included in the PRACA systems (or even 
PC ASS) and unavailable to the Program Office. 

• Every PRACA system is unique and designed primarily for Project/Element (subsystem domain) use. Program use 
of the systems and their data are mainly handled as design patches to the systems. 

• USA s ADAM is incomplete and unable to act as a data warehouse supporting cross database data mining. A 
proposed KSC/USA WebPCASS based upon the existing ADAM structure is currently proposed. This is a good first 
attempt to provide unified PRACA data and some associated information, but needs to go much farther. 

• Generating SR&QA reports is an extremely time and labor intensive activity requiring specialized knowledge and 
much massaging and cleaning of PRACA data. 

To improve Program- level access to the PRACA data, we recommend that the current PRACA system be replaced or 
significantly upgraded. 


Recommendations: 

• Clearly identify (list) the Program-level PRACA tasks from a Program-wide perspective 

• Establish requirements for a PRACA System that performs SSP level PRACA tasks (data retrieval, mining and 
trending needs). This action should be performed without consideration of current PRACA capabilities. 

• Design a PRACA System that satisfies these requirements. 

• Either a) Implement this new system or b) Initiate a modernization activity to upgrade the current PRACA systems 
and designs to satisfy the requirements. 

• Replace or enhance the existing WebPCASS proposal based upon the above decisions. 


PRACA Assessment Areas Recommendations 


As a foundation for the new PRACA system design, the team has identified specific deficiencies and recommended actions 
for each of the four assessment areas identified in our Study approach. It is important to note however, that we believe the 
Program-wide vision for PRACA (i.e„ what PRACA should be ) must precede system technology changes. 

With regard to the assessment area of functional capabilities and the upgrades recommended, it is our opinion that a PRACA 
Enhancement Project should 

• Satisfy all the Program Offices task-based requirements (see previous section); 

• Satisfy the Project/Element (subsystem domain) work flow management requirements; 

• Meet all NASA data security standards; 

• Increase the user base through ease of access and intuitive user interface; 

• Incorporate expert knowledge capture to assist in correct data interpretation and to reduce dependency on human 
corporate/ institutional knowledge; 

• Simplify system management and support requirements; 

• Integrate with other Shuttle data sources to enable Data Mining in support of Risk Assessments; 

• Provide advanced IT capabilities for Trending and Analysis in support of SRQ&MA requirements, 

Provide a migration path to a true safety and risk prognostics capability for the SSP. 

6. CONCLUSION 

The SSP PRACA System is an essential component to enable increased Shuttle safety and improved assessment of Shuttle 
readiness for flight. With the significant growth in the capabilities of Information Technology, the SSP PRACA System is 
poised to take advantage of the increased capabilities these advances provide. The SSP is motivated to increase the 
knowledge extraction capability of PRACA by using advanced IT tools for improved ease of access, greater breadth and 
depth of risk assessments, enhanced data quality and integrity, faster data mining and trending, and progression towards a 
true Safety and Risk Prognostic capability. 

This Study has identified several areas where improvements in technology or implementation can enable a significant SSP 
PRACA improvement. In addition, the SSP PRACA System enhancement activity is capable of benefiting from other 
development activities such as Design for Safety (DfS) Program technology insertion, leverage from Aviation Safety 
Program developments, and other basic information technology enhancements coming from the Intelligent Systems Program. 

We believe that an Agency-wide NASA/Industry team in conjunction with the SSP PRACA workforce can bring together the 
required expertise, knowledge base, and advanced IT capabilities necessary to achieve NASA s Information Management 
vision for PRACA. In so doing, PRACA will remain a critical and vital system, enabling a reduction in the risk and 
improvements in safety while supporting the Space Shuttle Program into the next decades. 
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